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ABSTRACT 


We have developed a prototype analysis system for remote viewings conducted 
apainst targets of interest. The system uses individual viewers’ performance histories 


in conjunction with current data to prioritize a set of possible interpretations of the 
site. 
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I INTRODUCTION 


Since 1973, when the investigations of the human information-accessing capability 
called remote viewing (RV) first began at SRI International,’* evaluating the quality of the 
information obtained has been a continuing challenge. In order to develop valid evaluation 


procedures, two basic questions must be addressed: 


(1) What constitutes the target? 


(2) What constitutes the response? 


If the RV task is research-oriented, the targets are known, and therefore can be 
precisely defined. In -oriented tasks, however, the targets are generally unknown 
and their descriptions are problematical. In both task domains, RV responses tend to consist of 
sketches and written phrases. A method to encode unambiguously this type of “natural 
language” is one of the unsolved problems in computer science, and there has been little progress 


to date. Thus, a complete definition of an RV response is also problematical. 


An -oriented RV task poses further problems. High-quality RV does 
not always provide useful . For example, the RV may provide additional support for 
information that has been verified from other sources, but provide no new information. In some 
cases, however, an overall low-quality RV may provide key elements that positively influence an 


analyst’s interpretation. 


Another characteristic of current laboratory analysis techniques is that they do 
not provide an a priori assessment of the RV quality. While this is not a problem in the 
laboratory, applications require such evaluation. An RV analyst cannot provide 

usefulness ratings from the RV alone; rather, the analyst must provide a priori 
probabilities that individual RV-response elements (or concepts) are present at the target site. It 
remains the responsibility of analyst to determine whether such data are ultimately 


useful. 

Analysis of laboratory RV has been a major part of the ongoing Cognitive 
Sciences Program.?-? For FY 1989, we focused on the development of a prototype analysis 
system that would provide the needed a priori assessments for tasking.t 


References are at the end of this report. 
t This report constitutes the deliverable for Statement of Work item 6.0.3. 
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The intelligence analyst, as opposed to an RV analyst, should construct such 
a list for each mission. While there may be considerable similarities between element lists for 
different missions, undoubtedly the lists will require specialization. In Section II-C below, we 
show the construction of one element list and how it can be applied to a set of 65 simulated 


operational targets. 


2. Analysis of Complete Responses 


Once an appropriate universal set of elements has been created, and fuzzy 
sets that define the target and the response have been specified, the comparison between them is 
Straightforward. We have defined accuracy as the percent of the target material that is described 
correctly by a response. Likewise, we have defined reliability (of the viewer) as the percent of 
the response that is correct. Although in the laboratory it is required to provide a posterior 
probability estimates of the target-response match, in an operational setting, this may be less 
important. All that is usually necessary is to describe the accuracy and reliability for complete 


responses, and for individual target elements of interest. These quantities for the jth sessions are 


>. Wi(Ry 1) Tix 
_ kal 


re - . (1) 
Dy WR} 
and kel 
n 
> Wi(Rj () Tie 
aie kel (2) 


nA 
> WkTjk 
kzl 


where the sum over & is called the sigma count in fuzzy set terminology, and is defined as the sum 
of the membership values (4) for the elements of the response, the target, or their intersection, 
and n is the number of possible elements as defined by the element list. A fuzzy intersection is 
defined as the minimum of the intersecting fuzzy set membership values. In this version of the 
definitions, we have allowed for the possibility of weighting the membership values, Wx, to 


provide mission-defined relevances. 


For the above calculation to be meaningful, the membership values for the targets 
must be similar in kind to those for the responses. For most mission~dependent specifications, 
this is generally not the case. The target membership values represent the degree to which a 
particular element is characteristic of the target, and the response membership values represent 
the degree to which the analyst is convinced that the given element is represented in the 


response. 
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Until RV abilities can encompass the recognition of elements as well as their degree of 
target characterization, we are required to modify the target fuzzy set. An analyst must decide 
upon a threshold above which an element is considered to be completely characteristic of the 
target site. In fuzzy set theory, this is called an a-cut: a technique to apply a threshold to the BL 
values such that if the original value exceeds it, reassign the value to 1, otherwise set it to 0. In 
this way, the analyst's subjectivity can be encoded in the response fuzzy set, and Equations 1 and 


2 remain valid. 


3. Analysis of an Individual Element 


Equations 1 and 2 can be simplified to provide an accuracy and reliability on an 
individual element basis instead for a complete response. For example, let N be the number of 
sessions against different targets that exist in a current archive for a specified viewer. Let « be an 
element in question (e.g., airport). Then the empirical probability that element e is in the target, 


given that the viewer said it was, is given by 


N 
R(e) =—, 3 
) N, (3) 
where Nc is the number of times that the individual was correct, and Nr is the number of times 
that element € was mentioned in the response. R(e) is also the reliability of the viewer for that 


specified element. 


To compute what chance guessing would be, we must know the occurrence rate 
of element e in the N sessions. Let No be the actual number of times element € was contained in 


the N targets. Then the chance-guessing empirical probability is given by 


N 
Ro(e) = a 


Ro(e) can also be considered as the guessing reliability (i.e., the reliability that would 
be observed if the viewer guessed e during every session). The more R(e) > Ro(e), the more 


reliable the individual is for the specified element. 


The empirical probability that the viewer said element e, given that it was in the 
target, is given by 
Ne 


A(e) ==. 
(€ Ne 


A(eé) is also the accuracy of the viewer for that specified element. 


AS a numerical example, suppose a single viewer participated in NV = 25 sessions. 


Let € = “airport.” Further suppose that No = 5 of the targets actually contained an airport. 
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Then, Ro(airport) = 0.20 is the chance probability (i.e., guessing airport during every session 
would only by 20 percent reliable). Assume that the viewer mentioned airport Nr = 6 times and 
was correct Ne = 4 times. Then this viewer’s reliability for airports is computed as R(airport) = 
0.67 > Ro(airport) = 0.20. The viewer’s accuracy for airports is computed as A (airport) = Ne/No 
= 0.80. Thus in this example, we can conclude that this viewer is reasonably accomplished at 


remote viewing an airport. 


B. Prototype Analysis System 


We assume that an : analyst has constructed a mission-dependent 
universal set of elements. We further assume that there are a number of competing 


interpretations of the target site in question. 


1. Target Templates 


The first step in our prototype analysis system is to define templates (i.e., 
general descriptions of classes of target types) of all competing target interpretations from the 
universal set of elements. For example, a class of target types could be a generic biological 
warfare (BW) facility. Exactly what the templates should represent is entirely dependent upon 
what kind of information is sought. Both the underlying universal set of elements and the 
templates must be constructed to be rich enough to allow for the encoding of all the information 
of intelligence interest. That is, if neither the set of elements nor the templates can meaningfully 
represent information about, say BW development sites, then it will be unreasonable to consider 
asking, “Does development of BW agents take place at the site?” Furthermore, a certain 
amount of atomization is necessary because such division into small units provides the potential] 
for interactions within the universal set of elements. If the profile of a BW facility consists of a 
single element, the template would be useless unless the response directly stated that particular 
element; rather, the profile should be constructed from groups of elemental features (e.g., 


biological, offensive, weapon, decontamination). 


There are two different ways to generate target templates. The most 
straightforward technique is also likely to be the most unreliable, because it relies on the analyst’s 
judgment of a single target type. With this method, the analyst, who is familiar with the 

problem at hand, simply generates membership values for elements from the 
universal set of elements based upon his or her general knowledge. Given the time and 
resources, the best way to generate template membership values is to encode known targets that 
are closely related (e.g, a number of known BW sites). Each template p is the average value 


across targets, and thus is more reliable. If it is known that some targets are more 
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“characteristic” of the target type than others, then a weighted average should be computed. In 


symbols, 
v 
>, Oni. k 
_ kel 
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where the sums are over the available targets that constitute the template, wx are the target 


weights, and the j1j,x are the assigned membership values for target k. 


2. Archival Database 


A critical feature of an analysis system for RV data is that along 
with the current RV data to be evaluated, the individual viewer’s past performance on an 
element-by-element basis must also be included. For example, if a viewer has been relatively 
unsuccessful at recognizing BW facilities, then a BW reference in the current data should not 


contribute much in the overall analysis. 


As ground truth becomes available for each session, a performance database 
should be updated for each viewer to reflect the new information This database should be a 
fuzzy set whose membership values for each element are the reliabilities computed from 


Equation 3. 


3. Optimized Probability List 


The goal of any RV analysis system is to provide an a priori 
prioritized and weighted list of target possibilities that results from a single remote viewing that is 
sensitive to the performance history of the viewer. Assuming that a template exists for each of 


the possible interpretations, an analyst should adhere to the following protocol: 


(1) Analyze the RV data by assigning a membership value (yu) for each element in the 
universal set of elements. Each p represents the degree to which the analyst is 
convinced that the particular element is included in the response. For example, 
suppose that the viewer said, “I perceive a BW facility.” Then uw(BW facility) = 1. 
Alternatively, suppose the viewer said, “I perceive glassware and smell organic 
chemicals.” In this case, p(BW facility) might be assigned 0.6. 


(2) Construct a crisp set, Re, as an a-cut of the original response set. By adopting a 
threshold of 0.5, for example, then the resulting crisp set contains only those 
elements that the analyst deems most likely as being present in the response. 


(3) Construct an effective response set, Re, as Re = Rc() Ra, where Ra is the reliability set 
drawn from the archival database. For example, suppose the original 
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(4) 


(5) 


assignment from the raw RV data was p(BW facility) = 0.6. Then after the a-cut 
with a threshold set at 0.5, n(BW facility) = 1.0. Suppose, however, that the viewer 
has been performing well on BW facilities and the archival database shows that 
Ra(BW facility) = 0.8. Thus, Re(BW facility) = 0.8. 


Using this effective response set, compute an accuracy and reliability in accordance 
with Equations 1 and 2. Then compute a figure-of-merit, Mj, for the jth competing 
interpretations as 


Mj=ajXrj. 
Of course, the accuracy and reliability use the effective response set from step 3 
above. 


Order the Ms from largest to smallest value. Since the figures—of—merit range in value 
from 0 to 1, they can be interpreted as relative probability values for each of the 
alternative target possibilities. soe 


By following such a protocol, an analyst can produce a list of target alternatives that is sensitive to 


the current remote viewing yet takes into consideration to the individual viewer's archival record. 


Cc. 


Partial Application of Analysis System to Existing Target Pool 


We have used an existing target pool (developed under a separate program) as a test 


bed for the analysis system described above. 


Criteria for Inclusion in the Target Pool 
Targets in this pool have the following characteristics: 


® Each target is within an hour and a half automobile drive of SRI International. 
e Each target simulates an operational site of interest. 


e Each target fits generally within one of five functional categories: Production, 
Recreation, Scientific, Storage, and Transportation. 


e Each target meets a consensus agreement of experienced RV monitors and 
analysts about inclusion in the pool. 


The pool consists of 65 targets. Initially, they were divided into 13 groups of five 


targets each, where each group contained one target from each of five functional categories. By 


carefully organizing the targets in this way, the maximum possible functional difference of the 


targets within each group was ensured. Table 1 shows a numerical listing of these targets. 
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. Transformer Station 
. Ballpark 

. Satellite Dish 

. Weapons Storage 

. Naval Fleet 

. Gravel Quarry 

. Swimming Pool 

. Observatory 

. Prison 

. Shipping and Receiving 
. Greenhouse 

. Picnic Area 

. Satellite Dishes 

. Paint Warehouse 

. Naval Air Station 

. Sugar Refinery 

. Playground 

. Aquarium 

. Drum Yard 

. Aircraft 

. Sewage Treatment Plant 
. Hoover Tower 


merwAQINMA BW HD we 


Table 1 


Numerical Listing of Targets 


. Space Capsule 

. Coastal Battery 

. Bay Area Rapid Transit 
. Salt Refinery 

. Candlestick Park 

. Solar Observatory 
. Food Terminal 

. Pedestrian Overpass 
. Electrical Plant 

. White Plaza 

. Space Shuttle 

. Coastal Battery 

. Train Terminal 

. Sawmill 

. Pond 

. Wind Tunnel 

. Grain Terminal 

. Submarine 

. Cogeneration Plant 
. Park 

. Linear Accelerator 
. Dump 


. Pump Station 

. Ice Plant 

. Caves/Cliffs 

. Bevatron 

. Barn 

. Golden Gate Bridge 
- Modern Windmills 

. Baylands Nature Preserve 
. Gas Plant 

. Auto Wreckers 

. Fishing Fleet 

. Radio Towers 

. Vineyard 

. Pharmaceutical Laboratory 
. Toxic Waste Storage 
. Airport 

. Car Wash 

. Old Windmill 

. Nuclear Accelerator 
. Reservoir 

. Train Station 


2. Fuzzy Set Element List 


In FY 1989, we developed a prototype analysis system for analyzing targets 
and responses in operational remote viewings. A list of elements, based on target function (i.e., 
the mission specification), is arranged in levels from relatively abstract (information poor) to the 
relatively complex (information rich). Having levels of elements is advantageous in that each can 


be weighted separately in the analysis. 


This universal set of elements (included as Appendix A) represents primary 
elements in the existing target pool of 65 targets. The set was derived exclusively from this 
known target pool. In an actual RV session, however, a viewer does not have access to the 
element list, and thus is not constrained to respond within its confines. An accurate RV analysis 
must include any additional] data that may be provided in the response; therefore, additional 
space has been provided on the analysis sheets (see Appendix A) to include elements that are 


part of the response but not initially included as part of the universal set. 


Approved For Release 2000/08/08 : CIA-RDP96-00789R002200540001-8 


Approved For Release 2000/08/08 : CIA-RDP96-00789R002200540001-8 


The target-dependent elements emphasize the site’s function, and use terms 
that are potentially universal across targets. We identified six element levels ranging from 
relatively information rich to relatively information poor: affiliation, function, attributes, 
modifiers, objects, and general/abstract. Because operational RV presupposes a certain level of 
ability on the part of the viewer, there are relatively few general/abstract elements included in our 
prototype analysis system. A description of some of the elements shown in Appendix A and a 


guide to their use are presented in Appendix B. 


3. Target Similarities 


In order to generate a demonstration target-type template using Equation 4, we 


first organized the 65 targets into clusters of similar types. 


We begin by defining the similarity between target j and target k (Sj,x) to be a 


normalized fuzzy set intersection between the two target sets; 


(> Wi(Tj 1) roi} 


is} 


Seog a (5) 
(> win) x Py wie) 
= 


iz 


By inspection, we see that Sj,x is also the figure~of-merit between target j and target k. 


For N targets there are N(N-1)/2 unique values (2080 for N=65) of Six. The 
value j and & that correspond to the largest value of Sj,x represent the two targets that are most 
functionally similar. Suppose another target m is chosen and Sn,j and Sim.k are computed. If 
both of these values are larger than Sm,p (for all p not equal to j or &) then target m is assessed to 
be most similar to the pair j,k. The process of grouping targets based on these similarities is 


called cluster analysis. 


Figure 1 shows the six clusters found from the cluster analysis of the 65 targets. * 
The numbers shown refer to the targets listed in Table 1, and the clusters are in close agreement 
with the original five categories used to select the targets. The point, however, is that a numerical 


algorithm is capable of dividing a set of targets into functional categories. 


In order to make the graphic output more meaningful, we used / - Sj,x in the analysis. 
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Cluster 1 
Recreation 


Cluster 2 
Transportation 


Cluster 3 
Weapons 


Cluster 4 
Technology 


Cluster 5 
Storage 


Cluster 6 
Production/Distribution 


Figure 1. Cluster Diagram for Simulated Operational Targets 


We used the technology cluster (i.e., number 4 in Figure 1) to apply Equation 4 
to construct a technology target template. Table 2 shows the targets in this cluster, where the 


horizontal lines indicate the subclustering within the technology group shown in Figure 1. 


10 
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Table 2 


Technology Cluster 
[ss [ses tones 


Transformer Station 
Modern Windmills 
Electrical Plant 
Cogeneration Plant 


Satellite Dish 
vs Satellite Dishes 
. Observatory ; 
- Solar Observatory 


=a Pharmaceutical Laboratory _* 


Nuclear Accelerator 
Linear Accelerator 
Bevatron 


Table 3 shows those elements that met or exceeded average membership values 
of 0.4 using Equation 4. 
Table 3 


Principal Elements Contained in the Technology Template 


[Finsion ie 
[Awe | 
<a 


Modifier 


Commercial/Private 


Research/Experimentation 


Electricity/Radio 


Objects High Technology Electronics 


Restricted Access 
Wires/Cables 


Abstract Activity—Passive 
Ambiance—Indoor 
Ambiance—Manmade 
Ambiance—Outdoor 


Size—Medium 
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_ As a self-consistency check, we included the technology template in the total 
target pool and recalculated the clusters. As expected, the technology template was included 


within the subgroup of targets 3 and 13, and well within the technology cluster as a whole. 


D. General Conclusions 


The goal of this effort was to develop an analysis system that would prove 
effective in providing a priori assessments of remote viewing tasks. If the proper 
mission—dependent universal set of elements can be identified, then, using a viewer-dependent 
reliability archive, data from a single remote viewing can be used to prioritize a set of alternative 


target templates so as to chose the most likely one for the mission. 
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UNIVERSAL SET OF ELEMENTS FOR ANALYSIS OF FUNCTION 
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AN ANALYST’S GUIDE TO THE UNIVERSAL SET OF ELEMENTS (VU) 


A. Introduction 


This appendix is intended to assist an analyst in using the universal set of elements 
shown in Appendix A. We developed six levels of elements ranging from relatively abstract 
(information poor) to the relatively complex (information rich). 


B. Element Levels and Their Use 


The task of the analyst is to assign a membership value between 0 and 1 to each 
individual element. For targets, a numerical value will be assigned on the basis of the presence 
or absence of each element in terms of functional importance. For responses, the numerical 
value will be assigned on the basis of the degree to which the analyst is convinced that the 


element is contained in the response. 


All subsequent commentary is referenced by the element numbers in Appendix A. 
Although each level may contain a number of elements, only those individual elements that may 
need explanation are listed below. 


1. Element Level—Affiliation 


“Affiliation” represents an advanced level of remote viewing functioning. 
Although we infrequently observe this advanced functioning, the data are valuable, and, 
therefore, are included. Elements in this level can be assigned membership values by asking the 


question, “Who owns the target?” There are only three “affiliation” elements: 


(1) Commercial/Private. 


(2) Government: Federal, state, or local governmental ownership (e.g., municipal 
utilities), but excluding military. 


(3) Military: military ownership as separate from the above governmental ownership 
(e.g., a Navy submarine). 
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2. 


Element Level—Function 


“Function” also represents an advanced level of remote viewing functioning, and 


it may represent the most important information with regard to overall function. Elements are 


assigned membership values by asking the question, “What is(are) the primary function(s) of the 


target?” There are 14 “function” elements, and a few require further explanation: 


(6) 


(8) 
(11) 
(13) 


(16) 


3. 


Distribution: the primary function is to receive and to transmit something (e.g., an 
electrical transformer station). 


Extraction: as in the extraction of minerals from the ground. 
Reception: the primary function is only to receive (e.g., a satellite tracking station). 


Refining: the primary function is to refine a raw material into an intermediate or 
finished product (e.g., a saw mill). 


Transmission: the primary function is only to transmit (e.g., a radio tower). 


Element Level—Attributes 


“Attributes” can be thought of as clarification for the “function” level. 


Elements are assigned membership values by asking a question similar to, “If the function of the 


target is production, then what is being produced?” There are 20 “attribute” elements, and the 


following require further explanation: 


(18) 
(20) 
(21) 
(23) 


(24) 
(29) 
(32) 
(33) 


Animals: animals only. 
Biology: the study of living things in general. 
Chemistry: also includes chemicals. 


Ecology: symbiotic systems in nature, as in ecological zones (e.g., the Bay Lands 
Nature Preserve). 


Energy: energy in a broad sense that also includes radio waves. 
Nature/Natural: general natural objects (e.g., plants and animals). 


Plants: plants only. 
Space exploration: general, includes all experimentation done in space. 


Elements 18 and 32 are given a membership value if the target/response is specifically oriented to 


one item. Otherwise element 29 should be assigned a value. 


4, 


Element Level—Modifiers 


“Modifiers” can be thought of as a clarification of the “attributes” level. 


Elements are assigned membership values by asking a question similar to, “If the function of the 


target is production, and vehicles are being produced, then what kind of vehicles are they?” 


There are 36 “modifiers” elements, and only element 66 requires further explanation: 


B-2 
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(66) 


5. 


Symbiotic: symbiotic relationships not subsumed under natural or ecology (e.g., a 
cogeneration plant). 


Element Level—Objects 


“Objects” contains specific elements not necessarily related to function. 


Elements are assigned membership values on the basis of the presence or absence of each object 


in terms of functional importance. There are 47 “objects” elements, and the following require 


further explanation: 


(77) 
(79) 
(88) 
(95) 


(116) 
(117) 
(118) 


(119) 


6. 


Catwalk: elevated walkway. 
Coastline: used only as coastline of an ocean. 
High-Technology Electronics: silicon-based technology. 


Port/Harbor: port should be marked as in port of departure (e.g., airport, train 
Station, seaport). ; 


Water-Bounded: only completely bounded bodies of water (e.g., pool or pond). 
Water—Canal: manmade. 


Water-Large Expanse: the San Francisco Bay should be marked as a large 
expanse. 


Water-River: also includes stream. 


_ Element Level—General/Abstract Items 


This level contains the most abstract elements. There are 31 elements, and the 


following require further explanation: 


(121) 


(122) 


(123) 
(128) 
(140) 
(141) 
(142) 
(143) 
(144) 
(145) 
(146) 


(148) 
(149) 
(150) 
(151) 


Activity-Active: predominant visually active (e.g., an accelerator is very active 
electromagnetically, but would be considered passive, because there is little visual 
activity); potential activity is considered as passive. 


Activity~Passive: predominant visually passive (e.g., a ballpark is passive most of 
the time). 


Activity-Flowing (Water, Air, etc.): can be natural (e.g. creek) or manmade. 
Ambience-Dangerous: perceived and/or physically dangerous. 

Colorful: to be used only if especially characteristic. 

Modern: to be used only if especially characteristic. 

Odd/Surprising: to be used only if especially characteristic. 

Old: to be used only if especially characteristic. 

Personnel~Few: 1 to 10 employees mostly full-time. 

Personnel-Many: 10 to 1000 employees mostly full-time. 


Personnel-None: no full-time employees, but occasional human attention is 
allowed. 


Size-Large (University Campus): represents a “campus” size area. 
Size-Medium (Building): size of typical single buildings. 
Size-Small (Human): typically, the size of a human (i.e., 6 feet) 
Dull: to be used only if especially characteristic of the color. 
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